Providing adaptive asynchronous interactions in extended reality environments

ABSTRACT

In one example, a method performed by a processing system including at least one processor includes rendering an extended reality environment, receiving a request from a user endpoint device of a first user in the extended reality environment to place an avatar of the first user in the extended reality environment, placing the avatar of the first user within the extended reality environment, detecting that a second user is attempting to interact with the avatar of the first user, detecting conditions surrounding the avatar of the first user, identifying, based on the conditions, a set of candidate avatar personas for the first user, selecting a first avatar persona from among the set of candidate avatar personas, and rendering the avatar of the first user with the first avatar persona in the extended reality environment.

This application is a continuation of U.S. patent application Ser. No.17/221,732, filed on Apr. 2, 2021, now U.S. Pat. No. 11,282,278, whichis herein incorporated by reference in its entirety.

The present disclosure relates generally to extended reality (XR)systems, and relates more particularly to devices, non-transitorycomputer-readable media, and methods for generating and displayingavatars in XR environments that allow for adaptive asynchronousinteractions among users.

BACKGROUND

Extended reality (XR) is an umbrella term that has been used to refer tovarious different forms of immersive technologies, including virtualreality (VR), augmented reality (AR), mixed reality (MR), cinematicreality (CR), and diminished reality (DR). Generally speaking, XRtechnologies allow virtual world (e.g., digital) objects to be broughtinto “real” (e.g., non-virtual) world environments and real worldobjects to be brought into virtual environments, e.g., via overlays orother mechanisms. XR technologies may have applications in fieldsincluding architecture, sports training, medicine, real estate, gaming,television and film, engineering, travel, and others. As such, immersiveexperiences that rely on XR technologies are growing in popularity.

SUMMARY

In one example, the present disclosure describes a device,computer-readable medium, and method for generating and displayingavatars in XR environments that allow for adaptive asynchronousinteractions among users. For instance, in one example, a methodperformed by a processing system including at least one processorincludes rendering an extended reality environment, receiving a requestfrom a user endpoint device of a first user in the extended realityenvironment to place an avatar of the first user in the extended realityenvironment, placing the avatar of the first user within the extendedreality environment, detecting that a second user is attempting tointeract with the avatar of the first user, detecting conditionssurrounding the avatar of the first user, identifying, based on theconditions, a set of candidate avatar personas for the first user,selecting a first avatar persona from among the set of candidate avatarpersonas, and rendering the avatar of the first user with the firstavatar persona in the extended reality environment.

In another example, a non-transitory computer-readable medium storesinstructions which, when executed by a processing system, including atleast one processor, cause the processing system to perform operations.The operations include rendering an extended reality environment,receiving a request from a user endpoint device of a first user in theextended reality environment to place an avatar of the first user in theextended reality environment, placing the avatar of the first userwithin the extended reality environment, detecting that a second user isattempting to interact with the avatar of the first user, detectingconditions surrounding the avatar of the first user, identifying, basedon the conditions, a set of candidate avatar personas for the firstuser, selecting a first avatar persona from among the set of candidateavatar personas, and rendering the avatar of the first user with thefirst avatar persona in the extended reality environment.

In another example, a device includes a processing system including atleast one processor and a computer-readable medium storing instructionswhich, when executed by the processing system, cause the processingsystem to perform operations. The operations include rendering anextended reality environment, receiving a request from a user endpointdevice of a first user in the extended reality environment to place anavatar of the first user in the extended reality environment, placingthe avatar of the first user within the extended reality environment,detecting that a second user is attempting to interact with the avatarof the first user, detecting conditions surrounding the avatar of thefirst user, identifying, based on the conditions, a set of candidateavatar personas for the first user, selecting a first avatar personafrom among the set of candidate avatar personas, and rendering theavatar of the first user with the first avatar persona in the extendedreality environment.

BRIEF DESCRIPTION OF THE DRAWINGS

The teachings of the present disclosure can be readily understood byconsidering the following detailed description in conjunction with theaccompanying drawings, in which:

FIG. 1 illustrates an example system in which examples of the presentdisclosure may operate;

FIG. 2 illustrates a flowchart of an example method for providingadaptive asynchronous interactions in extended reality environments inaccordance with the present disclosure; and

FIG. 3 depicts a high-level block diagram of a computing devicespecifically programmed to perform the functions described herein.

To facilitate understanding, identical reference numerals have beenused, where possible, to designate identical elements that are common tothe figures.

DETAILED DESCRIPTION

In one example, the present disclosure enhances social engagement amongusers in extended reality (XR) environments by generating and displayingavatars that allow users to interact in an adaptive asynchronous mannerwith other users. As discussed above, XR technologies allow virtualworld (e.g., digital) objects to be brought into “real” (e.g.,non-virtual) world environments and real world objects to be broughtinto virtual environments, e.g., via overlays or other mechanisms. Onespecific type of virtual world object is an avatar, or a digitalrepresentation of a person. The avatar may be used by the user toexplore the XR environment, to interact with objects in the XRenvironment, and to interact with avatars of other users in the XRenvironment. Thus, the avatar functions as a virtual persona for theuser within the XR environment.

As more and more interactions move to virtual environments including XRenvironments, there is a growing user demand for the ability to connectwith other users via virtual personas that can be accessed at any time,e.g., including when the users who are represented by the virtualpersonas may not be “online” (e.g., actively logged into or engaged withthe application that is rendering the virtual environment). Presently,options for asynchronous user interactions (i.e., interactions in whichthe two or more interacting users are online at different, potentiallynon-overlapping times) are limited. For instance, these options includestatic messages (e.g., email, voice messages, and the like) and moredynamic content (e.g., videos that may reenact an interaction) whichhave no anchors to particular locations within the XR environment.

Examples of the present disclosure allow users of XR environments tocreate a plurality of different avatar personas, where each avatarpersona may provide an interaction that is representative of a differentbehavior of the user. The XR system (e.g., an application serverrendering the XR environment) may then select one of a user's manyavatar personas for presentation based on different triggers within theXR environment, where the triggers may be location-based orcontext-based. For instance, if the user's avatar is participating in acasual meeting within the XR environment, then it may be appropriate forthe user's avatar to high-five another user's avatar as a form ofgreeting. However, if the meeting is a more formal meeting with Americanusers (e.g., a business meeting), then it may be more appropriate forthe user's avatar to shake hands with other user's avatars as a form ofgreeting. Similarly, if the meeting is a more formal meeting withJapanese users, then it may be more appropriate for the user's avatar tobow to the other user's avatars as a form of greeting. Thus, locationaland situational context may be considered when determining which avatarpersona of a plurality of avatar personas created for a given usershould be presented to other users when the given user is not activelyengaged in the extended reality environment. These and other aspects ofthe present disclosure are described in greater detail below inconnection with the examples of FIGS. 1-3.

Within the context of the present disclosure, an interaction that is“asynchronous” is understood to refer to an interaction that isindependent of time. For instance, in an asynchronous interactionoccurring in an XR environment, at least one of the parties (e.g., afirst party) participating in the interaction may not be “online” (e.g.,logged into or otherwise actively engaged with the XR environment) atthe time that the interaction is taking place. However, the interactionmay still be able to proceed as a natural exchange that appears to betaking place synchronously, in real time as to one or more otherparties, e.g., a second party.

Moreover, within the context of the present disclosure, an “avatar” isunderstood to refer to a digital representation of a user, where thedigital representation may have certain visual features (e.g., anappearance), audible features (e.g., a voice), and the like which may beselected by the user. For instance, the user may configure the avatar tolook and/or sound like the user, or the user may configure the avatar tolook and/or sound different from the user. In either case, the avatarmay function as a representation of the user in the XR environment. Anavatar “persona” is understood to refer to a behavior of the user'savatar, where the behavior may vary depending on location, company,and/or other context of the avatar. According to examples of the presentdisclosures, multiple different personas may be created for the sameavatar. Thus, the avatar may act or behave differently in differentsituations (while the appearance of the avatar may or may not changedepending upon the situation).

To further aid in understanding the present disclosure, FIG. 1illustrates an example system 100 in which examples of the presentdisclosure may operate. The system 100 may include any one or more typesof communication networks, such as a traditional circuit switchednetwork (e.g., a public switched telephone network (PSTN)) or a packetnetwork such as an Internet Protocol (IP) network (e.g., an IPMultimedia Subsystem (IMS) network), an asynchronous transfer mode (ATM)network, a wireless network, a cellular network (e.g., 2G, 3G, and thelike), a long term evolution (LTE) network, 5G and the like related tothe current disclosure. It should be noted that an IP network is broadlydefined as a network that uses Internet Protocol to exchange datapackets. Additional example IP networks include Voice over IP (VoIP)networks, Service over IP (SoIP) networks, and the like.

In one example, the system 100 may comprise a network 102, e.g., atelecommunication service provider network, a core network, or anenterprise network comprising infrastructure for computing andcommunications services of a business, an educational institution, agovernmental service, or other enterprises. The network 102 may be incommunication with one or more access networks 120 and 122, and theInternet (not shown). In one example, network 102 may combine corenetwork components of a cellular network with components of a tripleplay service network; where triple-play services include telephoneservices, Internet or data services and television services tosubscribers. For example, network 102 may functionally comprise a fixedmobile convergence (FMC) network, e.g., an IP Multimedia Subsystem (IMS)network. In addition, network 102 may functionally comprise a telephonynetwork, e.g., an Internet Protocol/Multi-Protocol Label Switching(IP/MPLS) backbone network utilizing Session Initiation Protocol (SIP)for circuit-switched and Voice over internet Protocol (VoIP) telephonyservices. Network 102 may further comprise a broadcast televisionnetwork, e.g., a traditional cable provider network or an internetProtocol Television (IPTV) network, as well as an Internet ServiceProvider (ISP) network. In one example, network 102 may include aplurality of television (TV) servers (e.g., a broadcast server, a cablehead-end), a plurality of content servers, an advertising server (AS),an interactive TV/video on demand (VoD) server, and so forth.

In one example, the access networks 120 and 122 may comprise broadbandoptical and/or cable access networks, Local Area Networks (LANs),wireless access networks (e.g., an IEEE 802.11/Wi-Fi network and thelike), cellular access networks, Digital Subscriber Line (DSL) networks,public switched telephone network (PSTN) access networks, 3^(rd) partynetworks, and the like. For example, the operator of network 102 mayprovide a cable television service, an IPTV service, or any other typesof telecommunication service to subscribers via access networks 120 and122. In one example, the access networks 120 and 122 may comprisedifferent types of access networks, may comprise the same type of accessnetwork, or some access networks may be the same type of access networkand other may be different types of access networks. In one example, thenetwork 102 may be operated by a telecommunication network serviceprovider. The network 102 and the access networks 120 and 122 may beoperated by different service providers, the same service provider or acombination thereof, or may be operated by entities having corebusinesses that are not related to telecommunications services, e.g.,corporate, governmental or educational institution LANs, and the like.

In accordance with the present disclosure, network 102 may include anapplication server (AS) 104, which may comprise a computing system orserver, such as computing system 300 depicted in FIG. 3, and may beconfigured to provide one or more operations or functions in connectionwith examples of the present disclosure for anchor caching for extendedreality applications. The network 102 may also include a database (DB)106 that is communicatively coupled to the AS 104.

It should be noted that as used herein, the terms “configure,” and“reconfigure” may refer to programming or loading a processing systemwith computer-readable/computer-executable instructions, code, and/orprograms, e.g., in a distributed or non-distributed memory, which whenexecuted by a processor, or processors, of the processing system withina same device or within distributed devices, may cause the processingsystem to perform various functions. Such terms may also encompassproviding variables, data values, tables, objects, or other datastructures or the like which may cause a processing system executingcomputer-readable instructions, code, and/or programs to functiondifferently depending upon the values of the variables or other datastructures that are provided. As referred to herein a “processingsystem” may comprise a computing device including one or moreprocessors, or cores (e.g., as illustrated in FIG. 3 and discussedbelow) or multiple computing devices collectively configured to performvarious steps, functions, and/or operations in accordance with thepresent disclosure. Thus, although only a single application server (AS)104 and single database (DB) are illustrated, it should be noted thatany number of servers may be deployed, and which may operate in adistributed and/or coordinated manner as a processing system to performoperations in connection with the present disclosure.

In one example, AS 104 may comprise a centralized network-based serverfor generating extended reality media. For instance, the AS 104 may hostan application that allows users to create multiple different avatarpersonas of themselves, where the multiple different avatar personas maybe used to interact with other users within an XR environment when theusers represented by the avatars are offline (e.g., not logged into orotherwise actively engaged in the XR environment). The multipledifferent avatar personas may be programmed to represent instances ofdifferent behaviors of the associated user. For instance, in oneexample, when a first user is offline and a second user wishes tointeract with the first user within the XR environment, the AS 104 mayselect an avatar persona from among the first user's multiple avatarpersonas to conduct an asynchronous interaction with the second user.The AS 104 may select the avatar persona from among the multiple avatarpersonas based on the conditions and context of the desired interaction,such as the relationship of the second user to the first user (e.g.,family, friends, professional acquaintances, etc.), the conditions ofthe location within the XR environment where the interaction will takeplace (e.g., crowded, noisy, etc.), and the like.

In one example, AS 104 may comprise a physical storage device (e.g., adatabase server), to store sets of different avatar personas for theusers of the XR environment, as discussed in greater detail below. Forinstance, the AS 104 may store an index, where the index maps each userin the XR environment to a plurality of different avatar personas forthat user. In one example, the index may further map each avatar personato a set of predefined actions and/or utterances, where each action orutterance in the set of predefined actions and/or utterances may betriggered by different locations, contexts, and/or events.

In a further example, the AS 104 may also store user profiles which mayspecify user preferences that can be used to filter a set of avatarpersonas for an asynchronous interaction. For instance, in one example,a user profile may specify, for each user: demographic information(e.g., age, gender, marital status, education, etc.), device information(e.g., whether the user uses a head mounted display, a mobile phone, atablet computer, or the like to render and display XR media, the typesof connections used by the device to access XR media such as cellular orWiFi, etc.), interests (e.g., favorite hobbies, sports teams, music,movies, etc.), usage history with respect to XR media (e.g., types ofdigital objects that the user has interacted with and/or ignored in thepast), and/or preferences (e.g., does not like swearing or violence).

A user profile may also specify restrictions on the types ofinteractions that may be rendered for the user. For instance, a parentmay configure a child's profile so that interactions which may beconsidered too violent or too scary (or which may include too muchstrong language) are prohibited from being rendered. In one example,profiles may be stored on an opt-in basis, i.e., a user may elect to nothave a profile. In a further example, the user profiles may be stored inencrypted form to protect any user information that may be deemedprivate.

In one example, the DB 106 may store the avatar personas, the index,and/or the user profiles, and the AS 104 may retrieve the avatarpersonas, the index, and/or user profiles from the DB 106 when needed.For ease of illustration, various additional elements of network 102 areomitted from FIG. 1.

In one example, access network 122 may include an edge server 108, whichmay comprise a computing system or server, such as computing system 300depicted in FIG. 3, and may be configured to provide one or moreoperations or functions for providing adaptive asynchronous interactionsin XR environments, as described herein. For instance, an example method200 for providing adaptive asynchronous interactions in XR environmentsis illustrated in FIG. 2 and described in greater detail below.

In one example, application server 104 may comprise a network functionvirtualization infrastructure (NFVI), e.g., one or more devices orservers that are available as host devices to host virtual machines(VMs), containers, or the like comprising virtual network functions(VNFs). In other words, at least a portion of the network 102 mayincorporate software-defined network (SDN) components. Similarly, in oneexample, access networks 120 and 122 may comprise “edge clouds,” whichmay include a plurality of nodes/host devices, e.g., computing resourcescomprising processors, e.g., central processing units (CPUs), graphicsprocessing units (GPUs), programmable logic devices (PLDs), such asfield programmable gate arrays (FPGAs), or the like, memory, storage,and so forth. In an example where the access network 122 comprises radioaccess networks, the nodes and other components of the access network122 may be referred to as a mobile edge infrastructure. As just oneexample, edge server 108 may be instantiated on one or more servershosting virtualization platforms for managing one or more virtualmachines (VMs), containers, microservices, or the like. In other words,in one example, edge server 108 may comprise a VM, a container, or thelike.

In one example, the access network 120 may be in communication with aserver 110. Similarly, access network 122 may be in communication withone or more devices, e.g., a user endpoint device 112. Access networks120 and 122 may transmit and receive communications between server 110,user endpoint device 112, application server (AS) 104, other componentsof network 102, devices reachable via the Internet in general, and soforth. In one example, user endpoint device 112 may comprise a mobiledevice, a cellular smart phone, a wearable computing device (e.g., smartglasses, a virtual reality (VR) headset or other types of head mounteddisplay, or the like), a laptop computer, a tablet computer, or the like(broadly an “XR device”). In one example, user endpoint device 112 maycomprise a computing system or device, such as computing system 300depicted in FIG. 3, and may be configured to provide one or moreoperations or functions in connection with examples of the presentdisclosure for providing adaptive asynchronous interactions in XRenvironments.

In one example, server 110 may comprise a network-based server forgenerating XR media. In this regard, server 110 may comprise the same orsimilar components as those of AS 104 and may provide the same orsimilar functions. Thus, any examples described herein with respect toAS 104 may similarly apply to server 110, and vice versa. In particular,server 110 may be a component of an XR system operated by an entity thatis not a telecommunications network operator. For instance, a providerof an XR system may operate server 110 and may also operate edge server108 in accordance with an arrangement with a telecommunication serviceprovider offering edge computing resources to third-parties. However, inanother example, a telecommunication network service provider mayoperate network 102 and access network 122, and may also provide an XRsystem via AS 104 and edge server 108. For instance, in such an example,the XR system may comprise an additional service that may be offered tosubscribers, e.g., in addition to network access services, telephonyservices, traditional television services, and so forth.

In an illustrative example, an XR system may be provided via AS 104 andedge server 108. In one example, a user may engage an application onuser endpoint device 112 (e.g., an “XR device”) to establish one or moresessions with the XR system, e.g., a connection to edge server 108 (or aconnection to edge server 108 and a connection to AS 104). In oneexample, the access network 122 may comprise a cellular network (e.g., a4G network and/or an LTE network, or a portion thereof, such as anevolved Uniform Terrestrial Radio Access Network (eUTRAN), an evolvedpacket core (EPC) network, etc., a 5G network, etc.). Thus, thecommunications between user endpoint device 112 and edge server 108 mayinvolve cellular communication via one or more base stations (e.g.,eNodeBs, gNBs, or the like). However, in another example, thecommunications may alternatively or additional be via a non-cellularwireless communication modality, such as IEEE 802.11/Wi-Fi, or the like.For instance, access network 122 may comprise a wireless local areanetwork (WLAN) containing at least one wireless access point (AP), e.g.,a wireless router. Alternatively, or in addition, user endpoint device112 may communicate with access network 122, network 102, the Internetin general, etc., via a WLAN that interfaces with access network 122.

In the example of FIG. 1, user endpoint device 112 may establish asession with edge server 108 for obtaining an XR media. For illustrativepurposes, the XR media may insert one or more digital objects into areal-time image stream of a real world scene to generate an XRenvironment. In this regard, an example XR environment 114 isillustrated in FIG. 1.

In one example, the XR environment 114 may be viewed by a user throughthe user endpoint device 112, e.g., on a display of a head mounteddisplay or mobile phone, or through a set of smart glasses. As the usermoves through the XR environment 114, the user endpoint device 112 (oralternatively the AS 104, edge server 108, or server 110) may detect oneor more virtual items and/or avatars with which the user may interact.For instance, in the example of FIG. 1, the user may interact withvirtual items 116 ₁-116 _(n) (hereinafter individually referred to as a“virtual item 116” or collectively referred to as “virtual items 116”)and/or avatar 118. Each virtual item 166 and avatar 118 may have a setof predefined actions that the user may take with respect to the virtualitem 116 or avatar 118. Furthermore, in the case of the avatar 118, theavatar 118 may also have a set of predefined actions or utterances thatthe avatar 118 may perform or speak when triggered by the user. With noloss of generality, another example may represent a virtual item 116 asa digital twin (or XR representation) of a real world object, such as areal tree 116 ₁ being mapped to a virtual tree at location 116 ₁ wherethe XR experience can activate or disable itself according to theavatars or to the index of avatars in the presence of the virtual item.

As an example, where the virtual item is a tree (e.g., virtual item 116₁), the user may be able to climb the tree, pick a piece of fruit fromthe tree, or the like. Where the virtual item is a mailbox (e.g.,virtual item 116 ₂), the user may be able to place a piece of mail inthe mailbox.

In the case of the avatar 118, the avatar 118 may represent a specificuser who may currently be offline (e.g., not currently logged into oractively engaged with the XR environment). However, the avatar 118 maybe programmed with a predefined set of actions and/or utterances (apersona) that allows the user to interact with the specific userrepresented by the avatar in an asynchronous manner. For instance, theavatar 118 may be triggered to say something to the user and/or toperform some action upon the user attempting an interaction with theavatar 118. For instance, if the user waves to the avatar 118 or sayshello, the avatar 118 may wave back or say hello to the user. The avatar118 may be further customized to resemble the specific user representedby the avatar and to speak and behave like the specific user. Thus, ifthe avatar represents a friend of the user, when the user attempts tointeract with the avatar 118, the avatar 118 may greet the user by nameor hug the user (or the user's avatar). This more personalizedinteraction may provide for a more immersive experience.

It should also be noted that the system 100 has been simplified. Thus,it should be noted that the system 100 may be implemented in a differentform than that which is illustrated in FIG. 1, or may be expanded byincluding additional endpoint devices, access networks, networkelements, application servers, etc. without altering the scope of thepresent disclosure. In addition, system 100 may be altered to omitvarious elements, substitute elements for devices that perform the sameor similar functions, combine elements that are illustrated as separatedevices, and/or implement network elements as functions that are spreadacross several devices that operate collectively as the respectivenetwork elements. For example, the system 100 may include other networkelements (not shown) such as border elements, routers, switches, policyservers, security devices, gateways, a content distribution network(CDN) and the like. For example, portions of network 102, accessnetworks 120 and 122, and/or Internet may comprise a contentdistribution network (CDN) having ingest servers, edge servers, and thelike for packet-based streaming of video, audio, or other content.Similarly, although only two access networks, 120 and 122 are shown, inother examples, access networks 120 and/or 122 may each comprise aplurality of different access networks that may interface with network102 independently or in a chained manner. In addition, as describedabove, the functions of AS 104 may be similarly provided by server 110,or may be provided by AS 104 in conjunction with server 110. Forinstance, AS 104 and server 110 may be configured in a load balancingarrangement, or may be configured to provide for backups or redundancieswith respect to each other, and so forth. Thus, these and othermodifications are all contemplated within the scope of the presentdisclosure.

To further aid in understanding the present disclosure, FIG. 2illustrates a flowchart of a method 200 for providing adaptiveasynchronous interactions in extended reality environments in accordancewith the present disclosure. In particular, the method 200 provides amethod by which a user in an extended reality environment may place anavatar in the extended reality environment in order to interactasynchronously with other users. In one example, the method 200 may beperformed by an XR server that is configured to generate digitaloverlays that may be superimposed over images of a “real world”environment to produce an extended reality environment, such as the AS104 or server 110 illustrated in FIG. 1. However, in other examples, themethod 200 may be performed by another device, such as the processor 302of the system 300 illustrated in FIG. 3. For the sake of example, themethod 200 is described as being performed by a processing system.

The method 200 beings in step 202. In step 204, the processing systemmay render an extended reality (XR) environment. As discussed above, theXR environment may be an environment that combines images or elements ofthe real world with digital or “virtual” elements. At least some of thevirtual elements may be interactive, such that a user may interact withthe virtual elements to trigger some action or event. A user may viewand interact with the XR environment using any type of device ofcombination of devices that is capable of displaying and exchangingsignals with the XR environment, including a mobile device such as amobile phone or tablet computer or a wearable device such as a headmounted display.

In one example, the processing system may render the XR environment bygenerating a digital overlay that is superimposed over a stream ofimages (e.g., video) of a real world environment. For instance, awearable device such as a head mounted display may present the overlayon a display while the user is viewing the real world environmentthrough the head mounted display. In another example, the processingsystem may generate an entirely digital environment in which certainelements of a real world environment (e.g., buildings, vehicles,geological features, landmarks, etc.) are recreated in digital form.

In step 206, the processing system may receive a request from a userendpoint device of a first user in the extended reality environment toplace an avatar of the first user in the extended reality environment.As discussed above, the avatar may comprise a digital representation ofthe first user. However, the avatar may not be simply a staticrepresentation of the first user; the avatar may be an interactivedigital persona that mimics or emulates the personality and behaviors ofthe first user. When the first user is logged into or actively presentand interacting within the XR environment, the first user may controlthe avatar. Thus, when the first user interacts with other users withinthe XR environment, the first user may actively control the utterancesand behaviors of the avatar in real time (i.e., as the avatar isinteracting with the other users).

However, the first user may wish to leave the avatar in the XRenvironment to interact with other users, even when the first user isnot logged into or actively present and interacting within the XRenvironment. Thus, in this case, the first user may select a locationwithin the XR environment and request that the processing system placethe avatar in that location so that other users may interact with theavatar. In this case, the first user may not be capable of activelycontrolling the utterances and behaviors of the avatar in real time(i.e., as the avatar is interacting with the other users).

In step 208, the processing system may place the avatar of the firstuser within the extended reality environment. As discussed above, in oneexample, the processing system may place the avatar of the first user ina location within the XR environment that is chosen by the first user.In some examples, however, the first user may not specify a location forthe avatar placement, or the requested location may be unavailable(e.g., placement of the avatar in the requested location may obstructother users' access to other interactive virtual elements within the XRenvironment, or placement of the avatar in the requested location mayrender the avatar difficult for other users to see). Thus, in suchcases, the processing system may automatically select a location for theplacement of the avatar. In one example, the processing system may placethe avatar as close as possible to a location requested by the firstuser (e.g., while minimizing any obstruction of other virtual elementsby the avatar).

As discussed above, in one example, a plurality of avatar personas maybe associated with the first user, where each avatar persona of theplurality of avatar personas may present an interaction that isrepresentative of a different behavior of the first user. For instance,a first avatar persona may present a casual social interaction that issuitable for greeting close friends (e.g., high-fiving, using slanglanguage, etc.), while a second avatar persona may present a more formalsocial interaction that is more suitable for greeting professionalacquaintances or individuals with whom the first user is less sociallyfamiliar (e.g., shaking hands, using more formal language, etc.). In oneexample, the request from the first user may identify which avatarpersona of the plurality of avatar personas the first user wishes tohave placed in the XR environment (e.g., as a default avatar persona).However, as discussed in further detail below, the avatar persona thatis presented to other users to represent the first user may be adaptedor changed over time based on changing circumstances and conditionswithin the XR environment.

In another example, the processing system may select which avatarpersona of the plurality of avatar personas to place for the first user(e.g., as a default avatar persona), based on the current conditionswithin the XR environment at the placement location. In one example, theconditions may include one or more of: the physical location of theplacement location (e.g., coordinates within a coordinate system), theorientation of the placement location, the environmental conditions(e.g., sunny, rainy, noisy, etc.) present at the placement location,time of day within the XR environment (e.g., morning hours. afternoonhours, evening hours, night hours, a particular hour or time (e.g., 1:00pm), etc.) and/or the social conditions (e.g., behaviors or demeanors ofother users) present at the placement location. At least some of theseconditions may be detected through the use of sensors (e.g., cameras,microphones, and/or the like) which may be used to gather data from theplacement location and provide the data to the processing system (oranother system) for further analysis.

In step 210, the processing system may detect that a second user isattempting to interact with the avatar of the first user, e.g., via auser endpoint device of the second user. At this time, the first usermay not be logged into or actively present and interacting in the XRenvironment. Thus, the avatar of the first user may conduct aninteraction with the second user in an asynchronous manner. That is,although the interaction may appear, from the second user's perspective,to be happening in real time (e.g., as if happening with a first userwho is logged into or actively present within the XR environment), theutterances and/or behaviors of the first user's avatar may be predefinedto at least some extent.

In one example, the processing system may detect the second userattempting to interact with the avatar of the first user when somesignal is received from the second user. For instance, the second user,e.g., via the user endpoint device of the second user, may click on orposition a pointer over the avatar of the first user, may speak anutterance that indicates that an interaction with the avatar of thefirst user is desired (e.g., “Hello, Eric” or “Excuse me, sir”), or maymove in a direction toward the avatar of the first user while the avatarof the first user is within the direction of the second user's gaze(e.g., as may be monitored by a head mounted display or other externalsensors).

In step 212, the processing system may detect the conditions surroundingthe avatar of the first user. As discussed above, the conditions mayinclude one or more of: the physical location of the placement location(e.g., coordinates within a coordinate system), the orientation of theplacement location, the environmental conditions (e.g., sunny, rainy,noisy, etc.) present at the placement location, the time of day, and/orthe social conditions (e.g., behaviors or demeanors of other users)present at the placement location. At least some of these conditions maybe detected through the use of sensors (e.g., cameras, microphones,and/or the like).

In step 214, the processing system may identify, based on the conditionsdetermined in step 212, a set of candidate avatar personas for theavatar of the first user (e.g., to replace the default avatar personaplaced in step 208). For instance, the conditions may serve as a filterfor determining which avatar personas of the plurality of avatarpersonas may be suitable or unsuitable for interacting with the seconduser at the current time. One or more machine learning and/or learningfrom demonstration techniques may be used to determine which types ofavatar personas (or more specifically, which types of action,utterances, and/or behaviors) may be suitable or unsuitable given aspecific set of conditions. Other examples may include the socialcompany as a combination of the first user and the second user in the XRexperience. For example, in one scenario, the second user may beaccompanied by her children, while the first user's avatar may beaccompanied by no other avatars. In this case, it may be appropriate topresent an informal, familiar avatar persona as the first user's avatarpersona. In another scenario, the second user may be alone, but thefirst user's avatar may be accompanied by other avatars or users whichmay trigger a more formal social environment. In another scenario,conditions may change as part of the XR experience itself, where boththe first user's avatar and the second user are participating in the“introduction” stage of a racing XR experience. However, after a periodof conversational time, the XR experience may transition to the“competition” stage, such that the avatar of the first user changesappearance or social mannerisms or detects similar changes from thesecond user.

As an example, if the conditions are noisy, it may not be appropriate toinclude an avatar persona in the set of candidate avatar personas whichwhispers or speaks in a very quiet voice. Similarly, if the second userappears to be upset (e.g., if audio of the second user appears to showthe second user crying, or if the avatar of the second user isfrowning), then it may not be appropriate to include an avatar personain the set of candidate avatar personas that is loud and boisterous. Asanother example, if the second user is determined to be a professionalacquaintance of the first user (e.g., based on recognition of the seconduser's user ID, comparison to a set of contacts for the first user, orthe like), then the set of candidate avatar personas may only includeavatar personas that are dressed in business attire (e.g., suits orbusiness casual attire, as opposed to jeans and t-shirts) and/or thatspeak and behave in a more formal demeanor.

In optional step 216 (illustrated in phantom), the processing system mayfilter the set of candidate avatar personas based on a preference of thesecond user, to generate a filtered set of candidate avatar personas forthe first user. For instance, a profile for the second user may includethe second user's preferences for interacting with other users in the XRenvironment. As an example, the second user may be a child whose parentshave configured parental control settings to control the child'sinteractions within the XR environment. Thus, the second user'spreferences may indicate that the second user does not wish to engage ininteractions that include swearing or violence. As a result, any avatarpersonas in the set of candidate avatar personas which may speak orbehave in a manner that is violent or that includes swearing may befiltered (i.e., removed) from the set of candidate avatar personas. Asanother example, the second user's preferences may indicate that thesecond user is an avid hockey fan. In this case, an avatar persona inthe set of candidate avatar personas that knows hockey statistics and/oris wearing the jersey of a professional hockey team (or is changeddynamically in the XR environment, e.g., showing the avatar taking theaction of flipping over a shirt revealing a hockey team logo or takingan action of putting on an outer jacket with the hockey team logo and soon) may be appropriate to keep for consideration in the set of candidateavatar personas, and may even be prioritized or ranked highly relativeto other avatar personas in the set of candidate avatar personas.

In step 218, the processing system may select a first avatar personafrom among the (optionally filtered) set of candidate avatar personas.In one example, the first avatar persona may be the avatar persona inthe set of candidate avatar personas that the processing systemdetermines is most appropriate for the conditions surrounding the avatarof the first user. For instance, the processing system may rank the setof candidate avatar personas in an order from most appropriate to leastappropriate for the conditions. The ranking may take into considerationany metadata associated with the plurality of avatar personas, where theprocessing system may attempt to match the metadata to metadata orkeywords describing the set of conditions. For instance, as discussedabove, if the conditions are noisy, then the processing system may rankan avatar persona whose metadata indicates that the avatar personaspeaks in a whisper relatively low. Similarly, if the profile for thesecond user indicates that the second user likes hockey, then theprocessing system may rank an avatar persona whose metadata indicatesthat the avatar persona is wearing a hockey jersey or can discuss hockeyrelatively high. In one example, the processing system may select theavatar persona with the highest ranking. In another example, theprocessing system may detect a pattern of behaviors from the second userthat will trigger selection of an avatar persona that is not related tothe most relevant interaction. For instance, in a gaming scenario or amalicious hacking scenario, the second user may repeat an interactionwith the first user's avatar to discover a weakness in the avatarindexing system or in an avatar index that is valid but is not intendedfor the second user's current conditions. In this situation, the systemmay default to a generic or incorrect avatar response to avoid ortemporarily suppress anomalous interactions of the second user (muchlike current password entry systems may block a user after a number ofconsecutive wrong entries). Other behaviors may also be enabled by thesystem, but the selection stage (e.g., step 218) may contain historicalconditions from the second user to assist in determining appropriateactions in this situation.

In step 220, the processing system may render the avatar of the firstuser with the first avatar persona in the extended reality environment.That is, the processing system may insert the avatar of the first userinto the XR environment so that the second user may interact with theavatar of the first user in a manner that is consistent with the firstavatar persona. The avatar of the first user may be rendered in such amanner that the avatar of the first user may replay a set of predefinedactions (e.g., high fiving, hand shaking, dancing, etc.) and/orpredefined utterances. In one example, the predefined actions and/orutterances may not necessarily be played in any sort of defined orcontinuous order. For instance, the avatar of the first user may have aset of predefined actions that the avatar of the first user can performand a set of predefined utterances that the avatar of the first user canspeak. Any one of these predefined actions or predefined utterances maybe selected by the processing system for rendering in response to somesort of trigger. The trigger may be an action or utterance of the seconduser. For instance, if the second user holds out a hand for a high five,then the processing system may control the avatar of the first user toreturn the high five. If the second user makes a statement about aparticular hockey team, then the processing system may control theavatar of the first user to respond with another statement about theparticular hockey team or about hockey in general.

The method 200 may thus return to step 210 and may continue to adapt theavatar of the first user (e.g., adapt the persona of the avatar of thefirst user) in response to the actions and/or utterances of the seconduser. This interaction may continue until the second user gives someindication that the interaction is over (e.g., the second user may walkaway, or say “goodbye,” or “I have to go now.”). Steps of the method 200may also be repeated for subsequent users who may wish to interact withthe avatar of the first user, and different avatar personas for theavatar of the first user may be selected for these subsequent usersbased on the conditions at the time of the interactions and/or thepreferences of the subsequent users, as discussed above.

Thus, examples of the disclosure allow a first user to interact with asecond user in an XR environment in an asynchronous manner. That is, thefirst user may be able to “interact” with the second user even when thefirst user is not online or actively engaged in the XR environment. Thisis made possible by the use of multiple different avatar personas forthe avatar of the first user, where each avatar persona of the multipledifferent avatar personas may be programmed to perform a set ofpredefined actions and/or speak a set of predefined utterances thatprovide an interaction that is representative of a different behavior ofthe first user. Thus, when the second user attempts to interact with thefirst user (who may be offline or not actively engaged in the XRenvironment), a processing system may select an appropriate avatarpersona for the first user, from among the multiple different avatarpersonas, to present to the second user. This provides for a morenatural and more immersive experience for users, even when some usersmay be offline. It should be noted that the second user may also beoffline or not actively engaged in the XR environment in this case. Forinstance, two avatars having respective sets of predefined actionsand/or utterances may interact with each other in an asynchronousmanner. Furthermore, lower ranked avatar personas of the avatar of thefirst user may be subsequently deployed if the second user returns toagain interact with the avatar of the first user in some later time.Selecting a different avatar persona of the avatar of the first user maybe more engaging for the second user to provide variability in theinteractions.

The ability to provide adaptive asynchronous interactions in XRenvironments may have a number of applications in the XR field. Forinstance, in one example, a user may take an avatar “selfie” and leavethe avatar at a particular location in an XR environment. In a heavilysocialized environment (e.g., where memes and sharing may beencouraged), these types of avatars may have predefined patterns ofactions that can be replayed or dismissed for the purposes ofentertainment or adaptation. The predefined patterns of actions mayembody the personality of the user who the avatar represents, therebyallowing the user to personalize the avatar. A first user might alsoidentify an avatar of a second user, where the avatar of the second useris performing some sort of action, and may “borrow” or request that theaction be applied to the first user's avatar.

In another example, a user may create an avatar to connect with aparticular action or location. In this case, the user may use localobjects in a location to capture actions for later replay. For instance,the user may create an avatar of himself performing a trick on askateboard, and leave the avatar at a skate park in the XR environment.In another example, a user's avatar may be created with superheropowers, and the user may point to different locations or objects in theXR environment to define jump points for the avatar.

In another example, a repeatable game scenario may be created foreducational or entertainment purposes. For instance, different avatarscould be created to act as different game characters, to performdifferent game actions (e.g., running back and forth, guarding anobject), to provide motivational support or hints, and the like. In oneparticular example, a virtual scavenger hunt could be created within theXR environment. In this case, certain asynchronous interactions withavatars may only be unlocked when a user has completed some other listof actions or interactions first.

In another example, an avatar may be created to function as a virtualtour guide or concierge. In this case, different actions, utterances,and/or behaviors for the avatar may be triggered at different locations.For instance, a virtual tour guide in an art museum may be programmedwith a first set of utterances (e.g., facts, trivia, answers tofrequently asked questions, etc.) that is triggered when the virtualtour guide is within x feet of a first painting and a second set ofutterances that is triggered when the virtual tour guide is within xfeet of a second painting. Thus, a user or business may be able tocreate a semi-knowledgeable presence with some limited interactionpatterns that can be placed at specific locations to assist, direct, oreducate other users at those locations.

As also discussed above, learning from demonstration techniques may alsobe used to help determine suitable or unsuitable behaviors, utterances,and actions for given conditions and contexts. In a further example,these techniques may also be used to learn how a particular user reactsto specific conditions and contexts, which may help to furtherpersonalize different avatars for the particular user. In a furtherexample, such techniques could be used to personalize a base or templateavatar for the particular user. For instance, a template avatar for“ninja” could be personalized to incorporate physical features,mannerisms, and/or speech patterns of the particular user.

In some examples, a user who has placed an avatar for asynchronousinteractions in an XR environment may receive notifications when otherusers interact with the avatar. For instance, the user may receive asummary or replay of the interactions. These notifications may help theuser to make decisions which may affect the avatar, such as adaptingcertain patterns of behavior to new users or contexts. For instance,when interacting asynchronously with specific other users, the user maywant the avatar to hug those other users after offering a high five.

Avatars may also be constructed from historical synchronous interactionsinvolving a user. Such avatars may emulate the user in a way that allowsfamily or friends to interact with the user when the user may no longerbe present or may only be capable of interacting in person in limitedways (e.g., as in the case of a user who may have neurodegenerativedisease or impairment due to an accident).

Although not expressly specified above, one or more steps of the method200 may include a storing, displaying and/or outputting step as requiredfor a particular application. In other words, any data, records, fields,and/or intermediate results discussed in the method can be stored,displayed and/or outputted to another device as required for aparticular application. Furthermore, operations, steps, or blocks inFIG. 2 that recite a determining operation or involve a decision do notnecessarily require that both branches of the determining operation bepracticed. In other words, one of the branches of the determiningoperation can be deemed as an optional step. However, the use of theterm “optional step” is intended to only reflect different variations ofa particular illustrative embodiment and is not intended to indicatethat steps not labelled as optional steps to be deemed to be essentialsteps. Furthermore, operations, steps or blocks of the above describedmethod(s) can be combined, separated, and/or performed in a differentorder from that described above, without departing from the examples ofthe present disclosure.

FIG. 3 depicts a high-level block diagram of a computing devicespecifically programmed to perform the functions described herein. Forexample, any one or more components or devices illustrated in FIG. 1 ordescribed in connection with the method 200 may be implemented as thesystem 300. For instance, a server (such as might be used to perform themethod 200) could be implemented as illustrated in FIG. 3.

As depicted in FIG. 3, the system 300 comprises a hardware processorelement 302, a memory 304, a module 305 for providing adaptiveasynchronous interactions in extended reality environments, and variousinput/output (I/O) devices 306.

The hardware processor 302 may comprise, for example, a microprocessor,a central processing unit (CPU), or the like. The memory 304 maycomprise, for example, random access memory (RAM), read only memory(ROM), a disk drive, an optical drive, a magnetic drive, and/or aUniversal Serial Bus (USB) drive. The module 305 for providing adaptiveasynchronous interactions in extended reality environments may includecircuitry and/or logic for performing special purpose functions relatingto the operation of a home gateway or XR server. The input/outputdevices 306 may include, for example, a camera, a video camera, storagedevices (including but not limited to, a tape drive, a floppy drive, ahard disk drive or a compact disk drive), a receiver, a transmitter, aspeaker, a display, a speech synthesizer, an output port, and a userinput device (such as a keyboard, a keypad, a mouse, and the like), or asensor.

Although only one processor element is shown, it should be noted thatthe computer may employ a plurality of processor elements. Furthermore,although only one computer is shown in the Figure, if the method(s) asdiscussed above is implemented in a distributed or parallel manner for aparticular illustrative example, i.e., the steps of the above method(s)or the entire method(s) are implemented across multiple or parallelcomputers, then the computer of this Figure is intended to representeach of those multiple computers. Furthermore, one or more hardwareprocessors can be utilized in supporting a virtualized or sharedcomputing environment. The virtualized computing environment may supportone or more virtual machines representing computers, servers, or othercomputing devices. In such virtualized virtual machines, hardwarecomponents such as hardware processors and computer-readable storagedevices may be virtualized or logically represented.

It should be noted that the present disclosure can be implemented insoftware and/or in a combination of software and hardware, e.g., usingapplication specific integrated circuits (ASIC), a programmable logicarray (PLA), including a field-programmable gate array (FPGA), or astate machine deployed on a hardware device, a computer or any otherhardware equivalents, e.g., computer readable instructions pertaining tothe method(s) discussed above can be used to configure a hardwareprocessor to perform the steps, functions and/or operations of the abovedisclosed method(s). In one example, instructions and data for thepresent module or process 305 for providing adaptive asynchronousinteractions in extended reality environments (e.g., a software programcomprising computer-executable instructions) can be loaded into memory304 and executed by hardware processor element 302 to implement thesteps, functions or operations as discussed above in connection with theexample method 200. Furthermore, when a hardware processor executesinstructions to perform “operations,” this could include the hardwareprocessor performing the operations directly and/or facilitating,directing, or cooperating with another hardware device or component(e.g., a co-processor and the like) to perform the operations.

The processor executing the computer readable or software instructionsrelating to the above described method(s) can be perceived as aprogrammed processor or a specialized processor. As such, the presentmodule 305 for providing adaptive asynchronous interactions in extendedreality environments (including associated data structures) of thepresent disclosure can be stored on a tangible or physical (broadlynon-transitory) computer-readable storage device or medium, e.g.,volatile memory, non-volatile memory, ROM memory, RAM memory, magneticor optical drive, device or diskette and the like. More specifically,the computer-readable storage device may comprise any physical devicesthat provide the ability to store information such as data and/orinstructions to be accessed by a processor or a computing device such asa computer or an application server.

While various examples have been described above, it should beunderstood that they have been presented by way of example only, and notlimitation. Thus, the breadth and scope of a preferred example shouldnot be limited by any of the above-described example examples, butshould be defined only in accordance with the following claims and theirequivalents.

What is claimed is:
 1. A method comprising: rendering, by a processing system including at least one processor, an extended reality environment; receiving, by the processing system, a request from a user endpoint device of a first user in the extended reality environment to place an avatar of the first user in the extended reality environment; placing, by the processing system, the avatar of the first user within the extended reality environment; detecting, by the processing system, at least one condition surrounding the avatar of the first user; identifying, by the processing system based on the at least one condition, a set of candidate avatar personas for the avatar of the first user; selecting, by the processing system, a first avatar persona from among the set of candidate avatar personas; and rendering, by the processing system, the avatar of the first user with the first avatar persona in the extended reality environment.
 2. The method of claim 1, wherein the set of candidate avatar personas comprises a subset of a plurality of avatar personas associated with the avatar of the first user, and wherein each avatar persona of the plurality of avatar personas associated with the avatar of the first user represents a different behavior of the avatar of the first user.
 3. The method of claim 2, wherein the different behavior comprises a predefined action to be performed by the each avatar persona in response to an occurrence of a trigger.
 4. The method of claim 2, wherein the different behavior comprises a predefined utterance to be spoken by the each avatar persona in response to an occurrence of a trigger.
 5. The method of claim 2, wherein the at least one condition comprises a location of the avatar of the first user in the extended reality environment.
 6. The method of claim 5, wherein the at least one condition comprises an orientation of the location of the avatar of the first user in the extended reality environment.
 7. The method of claim 2, wherein the at least one condition comprises a time of day in the extended reality environment.
 8. The method of claim 2, wherein the at least one condition comprises an environmental condition of the location of the avatar of the first user in the extended reality environment.
 9. The method of claim 2, wherein the at least one condition comprises at least one social condition of a location of the avatar of the first user in the extended reality environment.
 10. The method of claim 2, wherein the at least one condition comprises a demeanor of a second user interacting with the avatar of the first user.
 11. The method of claim 2, wherein the at least one condition comprises a relationship of a second user interacting with the avatar of the first user.
 12. The method of claim 1, wherein the rendering comprises controlling the avatar of the first user to carry out an interaction with a second user while the first user is offline.
 13. The method of claim 1, further comprising: filtering, by the processing system, the set of candidate avatar personas based on a preference of a second user interacting with the avatar of the first user to generate a filtered set of candidate avatar personas for the first user, wherein the first avatar persona is selected from among the filtered set of candidate avatar personas.
 14. The method of claim 13, wherein the filtering comprises removing, from the set of candidate avatar personas for the first user, any avatar personas representing behavior that the preference indicates the second user does not wish to see.
 15. The method of claim 13, wherein the filtering comprises removing, from the set of candidate avatar personas for the first user, any avatar personas speaking utterances that the preference indicates the second user does not wish to hear.
 16. The method of claim 1, wherein the rendering comprises controlling the avatar of the first user to function as a virtual tour guide within the extended reality environment.
 17. The method of claim 16, wherein the rendering further comprises controlling the avatar of the first user to speak an utterance that is triggered by a specific location within the extended reality environment.
 18. The method of claim 16, wherein the rendering further comprises controlling the avatar of the first user to perform an action that is triggered by a specific location within the extended reality environment.
 19. A non-transitory computer-readable medium storing instructions which, when executed by a processing system including at least one processor, cause the processing system to perform operations, the operations comprising: rendering an extended reality environment; receiving a request from a user endpoint device of a first user in the extended reality environment to place an avatar of the first user in the extended reality environment; placing the avatar of the first user within the extended reality environment; detecting at least one condition surrounding the avatar of the first user; identifying, based on the at least one condition, a set of candidate avatar personas for the first user; selecting a first avatar persona from among the set of candidate avatar personas; and rendering the avatar of the first user with the first avatar persona in the extended reality environment.
 20. A device comprising: a processing system including at least one processor; and a computer-readable medium storing instructions which, when executed by the processing system, cause the processing system to perform operations, the operations comprising: rendering an extended reality environment; receiving a request from a user endpoint device of a first user in the extended reality environment to place an avatar of the first user in the extended reality environment; placing the avatar of the first user within the extended reality environment; detecting at least one condition surrounding the avatar of the first user; identifying, based on the at least one condition, a set of candidate avatar personas for the first user; selecting a first avatar persona from among the set of candidate avatar personas; and rendering the avatar of the first user with the first avatar persona in the extended reality environment. 